A k%2DNearest Neighbor Based Multi%2DInstance Multi%2DLabel Learning Algorithm.

Multi%2Dlabel learning by exploiting label dependency.
In multi-label learning, each training example is associated with a set of labels and the task is to predict the proper label set for the unseen example. Due to the tremendous (exponential) number of possible label sets, the task of learning from multi-label examples is rather challenging. Therefore, the key to successful multi-label learning is how to effectively exploit correlations between different labels to facilitate the learning process. In this paper, we propose to use a Bayesian network structure to efficiently encode the conditional dependencies of the labels as well as the feature set, with the feature set as the common parent of all labels. To make it practical, we give an approximate yet efficient procedure to find such a network structure. With the help of this network, multi-label learning is decomposed into a series of single-label classification problems, where a classifier is constructed for each label by incorporating its parental labels as additional features. Label sets of unseen examples are predicted recursively according to the label ordering given by the network. Extensive experiments on a broad range of data sets validate the effectiveness of our approach against other well-established methods. Categories and Subject Descriptors I.2.6 [Computing Methodologies]: Learning—concept learning,
Multi%2Dinstance clustering with applications to multi%2Dinstance prediction.

Classifier Ensemble with Unlabeled Data

Feature selection for multi%2Dlabel naive Bayes classification.

Ml%2Drbf  RBF Neural Networks for Multi%2DLabel Learning.

MIMLRBF RBF neural networks for multi%2Dinstance multi%2Dlabel learning.

M3MIML A Maximum Margin Method for Multi%2Dinstance Multi%2Dlabel Learning.

MIML A Framework for Learning with Ambiguous Objects

Multi%2DLabel Learning by Instance Differentiation.

Solving multi%2Dinstance problems with classifier ensemble based on constructive clustering.

ML%2DKNN A lazy learning approach to multi%2Dlabel learning.
Abstract: Multi-label learning originated from the investigation of text categorization problem, where each document may belong to several predefined topics simultaneously. In multi-label learning, the training set is composed of instances each associated with a set of labels, and the task is to predict the label sets of unseen instances through analyzing training instances with known label sets. In this paper, a multi-label lazy learning approach named Mlknn is presented, which is derived from the traditional k-Nearest Neighbor (kNN) algorithm. In detail, for each unseen instance, its k nearest neighbors in the training set are firstly identified. After that, based on statistical information gained from the label sets of these neighboring instances, i.e. the number of neighboring instances belonging to each possible class, maximum a posteriori (MAP) principle is utilized to determine the label set for the unseen instance. Experiments on three different real-world multi-label learning problems, i.e. Yeast gene functional analysis, natural scene classification and automatic web page categorization, show that Ml-knn achieves superior performance to some well-established multi-label learning algorithms.


Multilabel Neural Networks with Applications to Functional Genomics and Text Categorization.
Abstract — In multi-label learning, each instance in the training set is associated with a set of labels, and the task is to output a label set whose size is unknown a priori for each unseen instance. In this paper, this problem is addressed in the way that a neural network algorithm named BP-MLL, i.e. Backpropagation for Multi-Label Learning, is proposed. It is derived from the popular Backpropagation algorithm through employing a novel error function capturing the characteristics of multi-label learning, i.e. the labels belonging to an instance should be ranked higher than those not belonging to that instance. Applications to two realworld multi-label learning problems, i.e. functional genomics and text categorization, show that the performance of BP-MLL is superior to those of some well-established multi-label learning algorithms.
Adapting RBF Neural Networks to Multi%2DInstance Learning.
Abstract. In multi-instance learning, the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. A bag is positive if it contains at least one positive instance, while it is negative if it contains no positive instance. In this paper, a neural network based multi-instance learning algorithm named RBF-MIP is presented, which is derived from the popular radial basis function (RBF) methods. Briefly, the first layer of an RBF-MIP neural network is composed of clusters of bags formed by merging training bags agglomeratively, where Hausdorff metric is utilized to measure distances between bags and between clusters. Weights of second layer of the RBF-MIP neural network are optimized by minimizing a sum-of-squares error function and worked out through singular value decomposition (SVD). Experiments on real-world multi-instance benchmark data, artificial multi-instance benchmark data and natural scene image database retrieval are carried out. The experimental results show that RBF-MIP is among the several best learning algorithms on multi-instance problems. Key words. Machine learning, Multi-instance learning, Radial basis function, Neu-ral networks, Hausdorff distance, Singular value decomposition, Principle component analysis, Content-based image retrieval 1.
A k%2Dnearest neighbor based algorithm for multi%2Dlabel classification.

Ensembles of Multi%2DInstance Neural Networks.

Improve Multi%2DInstance Neural Networks through Feature Selection.
Abstract. Multi-instance learning is regarded as a new learning framework where the training examples are bags composed of instances without labels, and the task is to predict the labels of unseen bags through analyzing the training bags with known labels. Recently, a multi-instance neural network named BP-MIP was proposed. In this paper, BP-MIP is improved through adopting two different feature selection techniques, i.e. feature scaling with Diverse Density and feature reduction with principal component analysis. In detail, before feature vectors are fed to a BP-MIP neural network, they are scaled by the feature weights found by running Diverse Density on the training data, or projected by a linear transformation matrix formed by principal component analysis. Experiments show that these feature selection mechanisms can significantly improve the performance of BP-MIP. Key words. Multi-instance learning, machine learning, neural networks, feature selection, backpropagation
Ensembles of Multi%2Dinstance Learners.
Abstract. In multi-instance learning, the training set comprises labeled bags that are composed of unlabeled instances, and the task is to predict the labels of unseen bags. Through analyzing two famous multi-instance learning algorithms, this paper shows that many supervised learning algorithms can be adapted to multi-instance learning, as long as their focuses are shifted from the discrimination on the instances to the discrimination on the bags. Moreover, considering that ensemble learning paradigms can effectively enhance supervised learners, this paper proposes to build ensembles of multi-instance learners to solve multi-instance problems. Experiments on a real-world benchmark test show that ensemble learning paradigms can significantly enhance multi-instance learners, and the result achieved by EM-DD ensemble exceeds the best result on the benchmark test reported in literature. 1
